Understanding Neural Networks through Representation Erasure

نویسندگان

  • Jiwei Li
  • Will Monroe
  • Daniel Jurafsky
چکیده

While neural networks have been successfully applied to many natural language processing tasks, they come at the cost of interpretability. In this paper, we propose a general methodology to analyze and interpret decisions from a neural model by observing the effects on the model of erasing various parts of the representation, such as input word-vector dimensions, intermediate hidden units, or input words. We present several approaches to analyzing the effects of such erasure, from computing its impact on evaluation metrics, to using reinforcement learning to erase the minimum set of input words in order to flip a neural model’s decision. In a comprehensive analysis of multiple NLP tasks from lexical (word shape, morphology) to sentence-level (sentiment) to document level (sentiment aspect), we show that the proposed methodology not only offers clear explanations about neural model decisions, but also provides a way to conduct error analysis on neural models.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Neuron Mathematical Model Representation of Neural Tensor Network for RDF Knowledge Base Completion

In this paper, a state-of-the-art neuron mathematical model of neural tensor network (NTN) is proposed to RDF knowledge base completion problem. One of the difficulties with the parameter of the network is that representation of its neuron mathematical model is not possible. For this reason, a new representation of this network is suggested that solves this difficulty. In the representation, th...

متن کامل

Handwritten Character Recognition using Modified Gradient Descent Technique of Neural Networks and Representation of Conjugate Descent for Training Patterns

The purpose of this study is to analyze the performance of Back propagation algorithm with changing training patterns and the second momentum term in feed forward neural networks. This analysis is conducted on 250 different words of three small letters from the English alphabet. These words are presented to two vertical segmentation programs which are designed in MATLAB and based on portions (1...

متن کامل

Rainfall-runoff modelling using artificial neural networks (ANNs): modelling and understanding

In recent years, artificial neural networks (ANNs) have become one of the most promising tools in order to model complex hydrological processes such as the rainfall-runoff process. In many studies, ANNs have demonstrated superior results compared to alternative methods. ANNs are able to map underlying relationship between input and output data without prior understanding of the process under in...

متن کامل

A Novel Fuzzy and Artificial Neural Network Representation of Overcurrent Relay Characteristics

Accurate models of Overcurrent (OC) with inverse time relay characteristics play an important role for coordination of power system protection schemes. This paper proposes a new method for modeling OC relays curves. The model is based on fuzzy logic and artificial neural networks. The feed forward multilayer perceptron neural network is used to calculate operating times of OC relays for various...

متن کامل

Multivalued Logic Mapping of Neurons in Feedforward Networks

A common view of feedforward neural networks is that of a black box since the knowledge embedded in the connection weights of a feedforward neural network is generally considered incomprehensible. Many researchers have addressed this deficiency of neural networks by suggesting schemes to obtain a Boolean logic representation for the output of a neuron based on its connection weights. However, t...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1612.08220  شماره 

صفحات  -

تاریخ انتشار 2016